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This  final  report  contains:  (1  a  list  of  technical  reports  which 
were  written  on  research  supported  by  this  project;  (2)  a  list  of 
publications  developed  from  these  technical  reports,  (3)  copies  of  our 
1982  and  1983  summary  of  research  program  written  for  the  ONR  Statistics 
and  Probability  Program  Summary  Report. 

Our  project  on  time  series  analysis  has  supported  collaboration 
with  Professor  H.  J.  Newton  (Texas  A&M)  and  Professor  Marcello  Pagano 
(Harvard  Biostatistics).  Its  overall  goal  is  to  develop  algorithms  that 
provide  researchers  in  diverse  scientific  fields  with  statistically  and 
computationally  efficient  techniques  for  univariate  and  multivariate  time 
series  analysis. 

The  general  ideas  and  concrete  results  of  the  research  contributions 
supported  by  ONR  in  this  project  seem  to  have  had  wide  impact  and 
applications.  Forecasting  and  spectral  analysis  (which  are  among  the  act 
problems  to  which  this  basic  research  project  has  made  significant 
contributions)  are  statistical  procedures  of  great  interest  and 
applicability  to  many  branches  of  the  Defense  Department. 
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N-21 

April  1981 

H.  J.  Newton  and 
Marcello  Pagano 

The  Finite  Memory  Prediction  of 
Covariance  Stationary  Time  Series 

N-  22 

April  1981 

H.  Joseph  Newton 
Marcello  Pagano 

The  Cholesky  Decomposition  Algorithm 
and  ARMA  Modeling 

N-23 

April  1981 

H.  Joseph  Newton 
Marcello  Pagano 

A  Method  for  Determining  Periods 
in  Time  Series 

N-24 

April  1981 

H.  Joseph  Newton 
Marcello  Pagano 

Minimum  Mean  Square  Error  Prediction 
of  Autoregressive  Moving  Average 

Time  Series 

N-25 

July  1981 

H.  Joseph  Newton 

On  Some  Numerical  Properties  of  ARMA 
Parameter  Estimation  Procedures 

N-26 

July  1981 

Emanuel  Parzen 

Autoregressi ve  Spectral  Estimation, 

Log  Spectral  Smoothing,  and  Entropy 

N-27 

July  1981 

Emanuel  Parzen 

Comments  on  "The  Measurement  of 

Linear  Dependence  and  Feedback 

Between  Multiple  Time  Series" 

N-28 

June  1982 

Emanuel  Parzen 

Autoregressive  Spectral  Estimation 

N-29 

June  1982 

Emanuel  Parzen 

Autoregressi ve  Spectral  Estimation  and 
Functional  Inference 

N-  30 

June  1982 

Emanuel  Parzen 

Maximum  Entropy  I nterpretati on  of 
Autoregressive  Spectral  Densities 

N-  31 

June  1982 

Emanuel  Parzen 

Time  Series  Model  Identification, 

Spectral  Estimation,  and  Functional 
Inference 
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N-  32 

June  1982 

H.  Joseph  Newton 

Simultaneous  Confidence  Bands  for 
Autoregressive  Spectra 

N-33 

July  1982 

Herbert  T.  Oavis 

H.  Joseph  Newton 
Marcello  Pagano 

A  Toeplitz  Gram-Schmidt  Algorithm 
for  Autoregressive  Modeling 

N-34 

October  1982 

H.  Joseph  Newton 

Modeling  Episodic  Time  Series 

N-35 

November  1982 

Emanuel  Parzen 

Time  Series  Model  Identification 
by  Estimating  Information 

N-36 

January  1983 

H.  Joseph  Newton 
Emanuel  Parzen 

Forecasting  and  Time  Series  Model 
Types  of  111  Economic  Time  Series 

N- 37 

March  1983 

Emanuel  Parzen 

Time  Series  ARMA  Model  Identification 
by  Estimating  Information 

Publication  of  Technical  Reports 


N-21  SIAM  Journal  of  Scientific  and  Statistical  Computing,  4,  1983, 
pp.  330-339. 

N-23  Journal  of  the  American  Statistical  Association,  78,  1983,  pp. 
152-157. 

N-25  Proceedings  of  the  13th  Symposium  on  the  Interface  of  Statistics 
and  Computing,  ed.  W.  Eddy,  Springer  Verlag:  New  York,  1981. 

N-26  IEEE  ASSP  Workshop  on  Spectral  EstimationI,  1981,  ed.  S.  Haykin. 

N-27  Journal  of  the  American  Statistical  Association,  77,  1982,  pp. 
320-32ZT  ~~  ~ 

N-28  Handbook  of  Statistics  III,  ed.  D.  Brillinger  and  P.  Krishnaiah, 
North  Holland:  Amsterdam  (in  press). 

N-29  Spectral  Analysi s  and  its  Use  in  Underwater  Acoustics, 

International  CoTiTerence  Proceedings,  AprirT982,  London,  ed. 

T.  S.  Durrani. 

N-30  Statistics  and  Probability  Letters,  1,  1982,  pp.  2-6. 

N- 31  Signal  Processing  in  the  Ocean  Environment  Workshop  Proceedings, 
Annapolis,  May,  1982,  ed.  E.  J.  Wegman  (in  press). 

N-32  Biometrika  1983  (in  press). 

N- 35  Studies  in  Econometrics,  Time  Series,  and  Multivariate  Statistics 
in  Honor  of  T.  W.  Anderson,  ed.  S.  Karlin,  T.  Amemiya,  L.  Goodman, 
1983  (in  press).  — 

N-36  Major  Time  Methods  and  Their  Relative  Accuracy,  by  S.  Makridakis 
et  al . ,  Wiley:  London,  1983  (in  press). 

N-37  Proceedings  of  15th  Symposium  on  the  Interface  of  Computer  Science 
and  Statistics,  North  Holland:  Amsterdam,  1984  (in  press). 
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Multiple  Time  Series  Modeling  ;ind  Time  Series  Theoretic 
PROJECT  TITLE:  Statistical  Methods 

CONTRACT  NO.:  N00014 -82-MP20001 

PRINCIPAL 

INVESTIGATOR:  Distinguished  Professor  Emanuel  Parzen 

Institute  of  Statistics 
Texas  A&M  University 
College  Station,  Texas  77843 
(713)  845-3188 


LONG  RANGE  SCIENTIFIC  OBJECTIVES:  To  develop  statistically  and  computationally 
efficient  parameter  estimation  and  model  identification  techniques  for  univariate 
and  multivariate  time  series  analysis,  and  to  obtain  a  unified  approach  to  time 
series  analysis,  which  serves  the  needs  of  each  of  its  diverse  fields  of  application. 


PROJECT  OBJECTIVES:  Time  series  analysis  is  regarded  as  composed  of  four  major 
problems  (model  identification,  spectral  analysis,  parameter  estimation,  and  fore¬ 
casting),  which  are  equivalent  in  the  sense  that  an  answer  to  any  one  of  these 
questions  generates  an  answer  to  the  other  three  questions.  These  problems  must  be 
solved  simultaneously,  since  the  "goodness"  of  model  fit,  parameter  estimates, 
spectral  estimates,  and  forecasts  are  essentially  inter-dependent,  and  must  employ 
simultaneously  both  the  time  domain  and  the  frequency  domain.  Spectral  estimation 
techniques  provide  new  functional  inference  methods  of  data  analysis. 

CURRENT  STATUS  OF  THE  PROJECT:  It  is  developing  approaches  for  attaining  a  goal  of 
time  series  analysis:  from  a  finite  sample,  identify  models  which  provide  efficient 
forecasts,  and  efficient  estimators  of  the  spectral  density  which  do  not  introduce 
spurious  spectral  peaks,  and  resolve  close  spectral  peaks. 

The  proposed  approach  first  determines  the  memory  type  of  a  time  series:  long, 
short,  or  no  (white  noise).  A  short  memory  time  series  is  modeled  by  an  approximating 
AR,  MA,  or  ARMA  scheme.  A  long  memory  time  series  Y(t)  is  modeled  as:  (1)  a  sum  of 
a  long  memory  signal  plus  a  short  memory  noise,  or  (2)  as  the  input  of  a  high-pass 
filter  whose  output  Y(t)  is  short  memory.  Approximating  autoregressive  schemes  play 
two  roles:  as  spectral  estimators,  and  as  diagnostic  tools  for  identification  of 
memory  type  and  ARMA  type.  Model  identification  criteria  are  motivated  through 
concepts  of  estimating  information.  Measures  of  information  divergence,  entropy, 
and  functional  inference  are  being  investigated,  yielding  a  new  proof  of  the 
maximum  entropy  character  of  autoregressive  density  estimators. 

SIGNIFICANT  ACCOMPLISHMENTS  Our  research  has  introduced  new  concepts  into  empirical 
time  series  analysis,  including:  CAT,  criterion  autoregressive  transfer  function, 
provides  order  determining  criteria  for  the  orders  of  approximating  autoregressive 
schemes,  developed  for  both  univariate  and  multivariate  time  series  [see  Newton  (1982)], 
and  provides  a  powerful  test  for  white  noise;  MEMORY  (no,  short,  and  long)  provides 
an  initial  classification  to  be  assigned  to  a  time  series  which  determines  our  strategy 
for  modeling  it  [Parzen  (1981)];  ARARMA  models  for  long-memory  time  series  which 
propose  that  the  transformation  from  long  memory  to  short  memory  be  a  best-lag 
non-stat ionary  autoregression  [Parzen  (1982)]. 

[1]  Parzen,  E.  (1981)  "Time  Series  Model  Identification  and  Prediction  Variance 

Horizon,"  Applied  Time  Series  Analysis  II,  ed.  by  D.  Findley,  Academic  Press:  New 
York,  415-447.  '  “ 

[2]  Parzen,  E.  (1982)  "ARARMA  Models  for  Time  Series  Analysis  and  Forecasting", 
Journal  of  Forecasting ,  Wiley:  London,  Vol.  I,  No.  1. 

[3]  Parzen,  E.  (1981)  Autoregressive  Spectral  Estimation,  Log  Spectral  Smoothing, 
and  Entropy.  IEEE  ASSP  First  Workshop  on  Spectral  Estimation. 

[4]  Newton,  H.  Joseph.  (1982)  Using  Periodic  Autoregressions  for  Multiple 
Spectral  Estimation.  Technomet  r lcs ,  May. 
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MULTIPLE  TIME  SERIES  MODELING  AND  TIME  SERIES 
THEORETIC  STATISTICAL  METHODS 

Emanuel  Parzen  -  Principal  Investigator 
Department  of  Statistics 
Texas  A&M  University 
College  Station,  Texas  77843 
409-845-3188 

Long^  Range  Scientific  Objectives:  Our  research  aims  to  achieve  the  basic  goals 
of  statistical  time  series  analysis,  which  is  to  develop  statistically  efficient 
and  computationally  effective  methods  for:  (1)  idcnt i f icat ion  of  models  for 
univariate  and  multivariate  time  series,  both  Gaussian  and  non-Caussian;  (2) 
estimat ion  of  their  time  domain  parameters;  (3)  spectral^  analysis;  (4) 
f  or  ecast ing . 

Project  Objectives:  This  project  seeks  to  develop,  both  for  theoretical  elegance 
and  piactical  guidance,  a  comprehensive  theory  of  model  identification  and 
spectral  estimation  which  clarifies  the  role  of  estimators  of  various  types  (non- 
parametric,  kernel,  maximum  entropy,  autoregressive,  ARMA,  ARIMA,  handlimited 
extrapolation,  cepstral,  etc.)-  This  project  investigates:  (1)  the  proposition 
that  the  key  ingredient  of  such  a  theory  is  the  notion  of  memory,  (2)  definitions 
of  memory,  (3)  various  methods  for  identification  of  memory  type,  (4)  various 
estimators  of  memory  index,  (5)  analogies  with  recent  research  on  estimation  of 
density-quantile  and  quantile  functions,  and  (6)  roles  of  statistical  computing 
in  time  series  analysis  in  two  important  ways:  (a)  to  rapidly  make  available  to 
the  broader  scientific  community  new  algorithms  for  time  series  analysis;  (b)  to 
make  old  theoretical  ideas  of  time  series  analysis  practically  useful  and  to 
stimulate  the  integration  of  old  and  new  techniques  of  time  series  analysis. 

Current  S t a_t us_o_f  t he_  P r o j e c t  _a nd_  Rec ent  S igni f ica nj^  Accomplishments :  To  identify 
models  for  time  series  and  dynamic  systems,  information  theoretic  ideas  and 
.approximation  theoretic  ideas  are  explored.  Models  for  time  series  Y(t)  can  be 
formulated  as  hypotheses  concerning  the  information  about  Y(t)  given  various  bases 
involving  past,  current,  and  future  values  of  Y(')  and  related  time  series  X(-). 

To  determine  sets  of  variables  that  are  sufficient  to  forecast  Y(t),  and  especially 
to  determine  an  ARMA  model  for  Y(t),  an  approach  is  developed  which  estimates  and 
compares  various  information  increments.  We  discuss  how  to  non-parametrically 
estimate  the  MA(“>)  representation,  and  use  it  to  form  estimators  of  the  many 
information  numbers  that  one  might  compare  to  identify  an  ARMA  model  for  a 
univariate  time  series.  We  discuss  how  to  estimate  an  index  of  regular  variation 
measuring  the  long  memory  behavior  of  the  spectral  density  at  zero  frequency.  We 
discuss  Toeplitz  Gram-Schmidt  algorithms  for  autoregressive  modeling. 

TRN-31  June  1982  Emanuel  Parzen  Time  Series  Model  Identification,  Spectral 
Estimation,  and  Functional  Inference 

TRN-32  June  1982  H.  Joseph  Newton,  Marcello  Pagano  Simultaneous  Confidence  Bands 
for  Autoregressive  Spectra 

TRN-33  July  1982  Herbert  T.  Davis,  H.  Joseph  Newton,  Marcello  Pagano  A  Toeplitz 
Gram-Schmidt  Algorithm  for  Autoregressive  Modeling 
TRN-34  October  1982  H.  Joseph  Newton  Modeling  Episodic  Time  Series 
TRN-35  November  1982  Emanuel  Parzen  Time  Series  Model  Identification  by 
Estimating  Information 

TRN-36  January  1983  H.  Joseph  Newton,  Emanuel  Parzen  Forecasting  and  Time  Series 
Model  Types  of  111  Economic  Time  Series 
TRN-37  March  1983  Emanuel  Parzen  Time  Series  ARMA  Model  Identification  By 
Estimating  Information 


